A Parametric Optimization Method for Machine Learning

The classiication problem of constructing a plane to separate the members of two sets can be formulated as a parametric bilinear program. This approach was originally created to minimize the number of points misclassiied. However, a novel interpretation of the algorithm is that the subproblems represent alternative error functions of the misclassiied points. Each subproblem identiies a speciied number of outliers and minimizes the magnitude of the errors on the remaining points. A tuning set is used to select the best result amoung the subproblems. A parametric Frank-Wolfe method was used to solve the bilinear subproblems. Computational results on a number of datasets indicate that the results compare very favorably with linear programming and simulated annealing approaches. The algorithm can be used as part of a decision tree algorithm to create nonlinear classiiers.